EchoPro Python Workflow

Import libraries and configure the Jupyter notebook

import matplotlib.pyplot as plt

# Python EchoPro
import EchoPro

# Allows us to grab the SemiVariogram class so we can use its models
from EchoPro.computation import SemiVariogram as SV

# obtain all visualization routines
from EchoPro.visualization import plot_layered_points, plot_kriging_results

# Allows us to easily use matplotlib widgets in our Notebook
%matplotlib widget

EchoPro.__version__

'0.2.0'

Set up EchoPro for a specific survey year

Initialize EchoPro object using configuration files

• initialization_config.yml – parameters independent of survey year

• survey_year_2019_config.yml – parameters specific to survey year

• source – Define the region of data to use e.g. US, CAN, US & CAN

• exclude_age1 – States whether age 1 hake should be included in analysis.

%%time
survey_2019 = EchoPro.Survey(init_file_path='../config_files/initialization_config.yml',
                             survey_year_file_path='../config_files/survey_year_2019_config.yml',
                             source=3, 
                             exclude_age1=True)

A full check of the initialization file contents needs to be done!
A check of the survey year file contents needs to be done!
CPU times: user 5.1 ms, sys: 515 µs, total: 5.62 ms
Wall time: 5.76 ms

Load and process input data

• This data is stored in survey_2019

%%time 
survey_2019.load_survey_data() #file_type='biological')

CPU times: user 1.39 s, sys: 11.8 ms, total: 1.4 s
Wall time: 1.4 s

survey_2019.specimen_df.head()

	sex	length	weight	age
haul_num
1	1	24.0	0.08	1.0
1	1	23.0	0.06	1.0
1	1	22.0	0.06	1.0
1	1	21.0	0.06	1.0
1	2	22.0	0.06	1.0

Compute the areal biomass density

• The areal biomass density is stored in survey_2019.bio_calc.transect_results_gdf as biomass_density_adult

%%time
survey_2019.compute_biomass_density()

CPU times: user 320 ms, sys: 439 µs, total: 320 ms
Wall time: 321 ms

survey_2019.bio_calc.transect_results_gdf.head()

	latitude	longitude	stratum_num	transect_spacing	biomass_density_adult	interval_area_nmi2	biomass	geometry
transect_num
1	34.397267	-121.143005	1	10.0	0.0	4.84267	0.0	POINT (-121.14301 34.39727)
1	34.397391	-121.133196	1	10.0	0.0	5.03849	0.0	POINT (-121.13320 34.39739)
1	34.397435	-121.123057	1	10.0	0.0	5.04074	0.0	POINT (-121.12306 34.39744)
1	34.397394	-121.112871	1	10.0	0.0	4.95015	0.0	POINT (-121.11287 34.39739)
1	34.397437	-121.102888	1	10.0	0.0	5.01398	0.0	POINT (-121.10289 34.39744)

print(f"Total Biomass Estimate without Kriging: {1e-6*survey_2019.bio_calc.transect_results_gdf.biomass.sum():.3f} kmt")

Total Biomass Estimate without Kriging: 1643.221 kmt

Jolly-Hampton CV Analysis

• Compute the mean of the Jolly-Hampton CV value on data that has not been Kriged

• Note: the algorithm used to compute this value is random in nature

%%time
CV_JH_mean = survey_2019.run_cv_analysis(kriged_data=False)

CPU times: user 1.24 s, sys: 257 µs, total: 1.24 s
Wall time: 1.24 s

print(f"Mean Jolly-Hampton CV: {CV_JH_mean:.4f}")

Mean Jolly-Hampton CV: 0.1328

Obtain Kriging Mesh Data

Access Kriging mesh object

• Reads mesh data files specified by survey_2019

krig_mesh = survey_2019.get_kriging_mesh()

Plot the transects and smoothed isobath contour, and optionally the mesh points

• plot_layered_points generates an interactive map using the Folium package

• Transect points are represented by a changing color gradient

• Smoothed contour points (200m isobath) are in blue

• Here we choose not to plot the mesh points to prevent this notebook from getting too big. To plot the mesh points, omit the plot_mesh_points argument or use plot_mesh_points=True. Mesh points will be in gray

fmap = plot_layered_points(krig_mesh, plot_mesh_points=False)

fmap

Make this Notebook Trusted to load map: File -> Trust Notebook

Apply coordinate transformations

• Longitude transformation

• Lat/Lon to distance

Transect points

krig_mesh.apply_coordinate_transformation(coord_type='transect')

Mesh points

krig_mesh.apply_coordinate_transformation(coord_type='mesh')

# plot the transformed mesh points 
plt.plot(krig_mesh.transformed_mesh_df.x_mesh, 
         krig_mesh.transformed_mesh_df.y_mesh, 'r*', markersize=1.25)
plt.show()

Compute biomass density Semi-Variogram and fit a model

• Compute the normalized semi-variogram using the areal biomass density

• Fit a model to the semi-variogram values

Compute the semi-variogram

Initialize semi-variogram calculation

• Transformed transect points

• Parameters specific to semi-variogram algorithm

semi_vario = survey_2019.get_semi_variogram(
    krig_mesh,
    params=dict(nlag=30, lag_res=0.002)
)

Compute the normalized semi-variogram

%%time
semi_vario.calculate_semi_variogram()

CPU times: user 3.62 s, sys: 2.03 s, total: 5.65 s
Wall time: 3.01 s

Fit a model to the semi-variogram

• A widget to easily fit the model

semi_vario.get_widget()

Perform Ordinary Kriging of biomass density

• transformed mesh points

• semi-variogram model

• areal biomass density

Initialize Kriging routine

kriging_params = dict(
    # kriging parameters
    k_max=10,
    k_min=3,
    R=0.0226287,
    ratio=0.001,
    
    # parameters for semi-variogram model
    s_v_params={'nugget': 0.0, 'sill': 0.95279, 'ls': 0.0075429,
                'exp_pow': 1.5, 'ls_hole_eff': 0.0},
    
    # grab appropriate semi-variogram model
    s_v_model=SV.generalized_exp_bessel
)

# uncomment to use widget values 
# kriging_params.update(semi_vario.get_params_for_kriging())

# initalize kriging routine
krig = survey_2019.get_kriging(kriging_params)

Perform Kriging

• Generates total biomass at mesh points

• Computes the total biomass estimate of Kriging output in kmt, where kmt is $10^3$ metric tons, which is equivalent to $10^6$ kg

%%time
krig.run_biomass_kriging(krig_mesh)

CPU times: user 5.03 s, sys: 3.2 s, total: 8.23 s
Wall time: 3.2 s

krig_results = survey_2019.bio_calc.kriging_results_gdf

Convert from kg to kmt

krig_results.biomass = 1e-6 * krig_results.biomass

print(f"Total Kriged Biomass Estimate: {krig_results.biomass.sum():.3f} kmt")

Total Kriged Biomass Estimate: 1725.033 kmt

Plot Kriged Biomass estimate in kmt

Here we use the argument greater_than_0=True to limit the mesh biomass points that are plotted to ones with biomass value > 0, to prevent this notebook from getting too big. To plot all mesh points, omit the greater_than_0 argument or use greater_than_0=False.

plot_kriging_results(krig_results, krig_field_name="biomass", greater_than_0=True)

Make this Notebook Trusted to load map: File -> Trust Notebook